Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial.

BACKGROUND: Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS: Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS: LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS: LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care.

Background: Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects. Method: A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate. Results: Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience. Conclusion: This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Using treatments with known positive impact in community-based mental health programs is challenging. Many studies suggest leaders of these programs can help. Similarly, certain features of community-based programs can also be helpful. This case study of an outpatient mental health clinic provides rich descriptions of actions leaders took that shaped the environment in their program and helped improve the use of a treatment with known positive impact. This case study can serve as a practical guide for leaders to reference when aiming to improve the use of treatments with known impact in their own programs.

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health.

OBJECTIVE: Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics. METHOD: In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline. RESULTS: A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023). CONCLUSION: Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services. CLINICAL TRIAL REGISTRATION INFORMATION: Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

Applying after-action reviews to child and family teams to improve mental health service linkage within child welfare services: a study protocol.

BACKGROUND: Half of child-welfare-involved children and adolescents meet the criteria for at least one mental health diagnosis. This project proposes to improve successful mental health service linkage in child welfare services (CWS) by adapting and testing the after-action review (AAR) team effectiveness intervention to augment the child and family team (CFT) services' intervention. Despite being both required and a collaborative approach to service planning, CFT meetings are implemented with questionable fidelity and consistency, rarely including the voice of children and families as intended. METHODS: Using a parallel group trial design, with non-equivalent comparison groups, and qualitative and quantitative methodology, this study will tailor and assess the impact of the AAR on enhancing CFT outcomes. The authors will conduct a qualitative needs assessment targeting the ongoing implementation of the CFT services intervention in a large, publicly funded, CWS system. A qualitative inquiry consisting of interviews and focus groups with key stakeholders will result in the preparation of an action plan to address identified gaps between the current and desired CFT services intervention outcomes. The AAR implementation strategy will be adapted and tailored to address the CFT services' intervention needs. To test the effectiveness of the AAR on improving outcomes associated with the CFT services intervention, we will utilize blocked randomization of four CWS caseworkers from two CWS system regions to either the intervention condition (CFT + AAR) or standard implementation (CFT as usual). The authors will collect data from the CWS caseworkers and additional CFT members via web-based surveys. Mechanisms of the AAR team effectiveness intervention for CFT implementation will be assessed. DISCUSSION: By inclusion of child and family voice, the AAR-enhanced CFT should lead to increased fidelity to the CFT intervention and greater levels of parental satisfaction with the service and shared decision-making, thus resulting in enhanced follow-through with service plans and linkage to mental health treatment services for children. The knowledge gained by this randomized clinical trial has the potential to benefit service delivery and integration for CWS leaders, caseworkers, formal and informal CFT member support persons, parents/caregivers, and children with open cases. Improving intervention effectiveness, both at the system and family levels, is crucial for practice efficiencies and improved child and family outcomes. TRIAL REGISTRATION: NCT05629013. Approval date: November 28, 2022 (version 1). TRIAL SPONSOR: University of California, San Diego. RESPONSIBLE PARTY: Danielle Fettes.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis.

BACKGROUND: Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility. METHODS: Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods. RESULTS: Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools. CONCLUSIONS: Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones. TRIAL REGISTRATION: ClinicalTrials NCT03719651, 25 October 2018.

"Change Doesn't Happen by Itself": A Thematic Analysis of First-Level Leaders' Experiences Participating in the Leadership and Organizational Change for Implementation (LOCI) Strategy.

The Leadership and Organizational Change for Implementation (LOCI) strategy is a multifaceted implementation strategy that aims to support successful evidence-based practice (EBP) implementation by fostering effective general leadership, implementation leadership, and implementation climate. How implementation strategies are experienced by participants is important for their utilization and effectiveness in supporting EBP implementation. The current study is the first in-depth qualitative study exploring first-level leaders' experiences of participating in the LOCI strategy. Data were collected as part of a trial where Norwegian child and adult mental health outpatient clinics implemented EBPs for posttraumatic stress disorder (PTSD). Eleven first-level leaders from adult and child clinics participated in semi-structured interviews after completing the LOCI strategy. Data were analyzed through reflexive thematic analysis. The analysis generated four themes related to leaders' experiences of participating in the LOCI strategy: (1) structuring the EBP implementation, (2) taking responsibility for the EBP implementation, (3) interacting with others about the EBP implementation, and (4) becoming aware of EBP implementation and their own leadership. Most participants experienced the LOCI strategy as beneficial for implementing EBPs for PTSD in their clinic. The strategy succeeded in raising awareness of leadership for EBP implementation, and simultaneously provided participants with tools and support for leading the implementation in their clinic. Two participants experienced LOCI as less beneficial than the others. Our results support the strategy's potential to engage and empower first-level leaders to get involved in implementation processes and point to important challenges for future research on implementation strategies.

Characterization of multilevel influences of mental health care transitions: a comparative case study analysis.

PURPOSE: Mental health care transitions are increasingly prioritized given their potential to optimize care delivery and patient outcomes, especially those focused on the transition from inpatient to outpatient mental health care. However, limited efforts to date characterize such mental health transition practices, especially those spanning multiple service setting contexts. Examination of key influences of inpatient to outpatient mental health care transitions across care contexts is needed to inform ongoing and future efforts to improve mental health care transitions. The current work aims to characterize multilevel influences of mental health care transitions across three United States-based mental health system contexts. METHODS: A comparative multiple case study design was used to characterize transition practices within the literature examining children's, non-VA adult, and VA adult service contexts. Andersen's (1995) Behavioral Health Service Use Model was applied to identify and characterize relevant distinct and common domains of focus in care transitions across systems. RESULTS: Several key influences to mental health care transitions were identified spanning the environmental, individual, and health behavior domains, including: community capacity or availability, cross-system or agency collaboration, provider training and experience related to mental health care transitions, client care experience and expectations, and client clinical characteristics or complexity. CONCLUSIONS: Synthesis illustrated several common factors across system contexts as well as unique factors for further consideration. Our findings inform key considerations and recommendations for ongoing and future efforts aiming to plan, expand, and better support mental health care transitions. These include timely information sharing, enhanced care coordination and cross setting and provider communication, continued provider/client education, and appropriate tailoring of services to improve mental health care transitions.

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate.

BACKGROUND: This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate. METHODS: A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate. RESULTS: After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI. CONCLUSIONS: The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation. TRIAL REGISTRATION: Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .

Implementation outcomes from a pilot of "Access to Tailored Autism Integrated Care" for children with autism and mental health needs.

LAY ABSTRACT: Children with autism frequently experience co-occurring mental health needs. The "Access to Tailored Autism Integrated Care (ATTAIN)" model was co-created with caregivers, pediatric providers, and health care leaders to identify mental health needs and link to mental health care for autistic children. This article describes outcomes from a pilot study of Access to Tailored Autism Integrated Care with 36 pediatric primary care providers from seven clinics within three healthcare systems. Providers participated in an initial Access to Tailored Autism Integrated Care training and received ongoing online support over 4 months with autistic patients ages 4-16 years old. Survey and interview assessments measured provider perceptions of feasibility, acceptability, and intentions to continue using Access to Tailored Autism Integrated Care after the pilot. Providers reported that Access to Tailored Autism Integrated Care was feasible, acceptable, that the initial training was helpful in their implementation but that more specific and tailored implementation support was needed. Results show that Access to Tailored Autism Integrated Care is a promising model to support mental health screening and linkage for children with autism in primary care. Findings provide information on specific areas of the Access to Tailored Autism Integrated Care model that could be benefit from additional refinement to support more widespread use in primary care settings.

Assessing readiness factors for implementation of LGBTQ+ affirmative primary care initiatives: Practice implications from a mixed-method study.

Introduction: Access and utilization barriers in primary care clinics contribute to health disparities that disproportionately affect lesbian, gay, bisexual, transgender, and queer (LGBTQ+) people. Implementing inclusive practice guidelines in these settings may decrease disparities. The purpose of this exploratory/developmental study is to identify key issues affecting the readiness of primary care clinics to implement such guidelines. Methods: Using a concurrent mixed-method research design, we conducted surveys, interviews, and focus groups with 36 primary care personnel in clinics in New Mexico, USA, to examine readiness to implement LGBTQ+ inclusive guidelines, analyzing factors affecting motivation, general organizational capacity, and innovation-specific capacity. We supplemented these data by documenting LGBTQ+ inclusive policies and practices at each clinic. We undertook descriptive analyses and between-subscale comparisons controlling for within-rater agreement of the survey data and iterative coding and thematic analysis of the qualitative data. Results: Quantitatively, participants reported significantly more openness toward adopting guidelines and attitudinal awareness for developing LGBTQ+ clinical skills than clinical preparedness, basic knowledge, and resources to facilitate implementation. Six themes derived from the qualitative findings corroborate and expand on these results: (1) treating all patients the same; (2) addressing diversity in and across LGBTQ+ populations; (3) clinic climates; (4) patient access concerns; (5) insufficient implementation support; and (6) leadership considerations. Conclusion: This study demonstrates that personnel in primary care clinics support initiatives to enhance service environments, policies, and practices for LGBTQ+ patients. However, drawing on Iris Young's theory of structural injustice, we found that neutralizing discourses that construct all patients as the same and time/resource constraints may diminish motivation and capacity in busy, understaffed clinics serving a diverse clientele and reinforce inequities in primary care for LGBTQ+ people. Efforts are needed to build general and innovation-specific capacities for LGBTQ+ initiatives. Such efforts should leverage implementation teams, organizational assessments, education, leadership support, community engagement, and top-down incentives.

Provider REport of Sustainment Scale (PRESS): development and validation of a brief measure of inner context sustainment.

BACKGROUND: Implementation scientists and practitioners often rely on frontline providers for reporting on implementation outcomes. Furthermore, measures of sustainment are few, and available sustainment measures are mainly setting or evidenced-based practice (EBP) specific, require organizational and system-level knowledge to complete, and often lack psychometric rigor. The aim of this study was to develop a brief, pragmatic, and generalizable measure for completion by frontline service providers of the implementation outcome, sustainment. METHODS: We utilized a Rasch measurement theory approach to scale the development and testing of psychometric parameters. Sustainment items were developed to be relevant for direct service providers to complete. In order to promote generalizability, data were collected and items were tested across four diverse psychosocial evidence-based practices (motivational interviewing [MI], SafeCare®, classroom pivotal response training [CPRT], and an individualized mental health intervention for children with autism spectrum disorder [AIM-HI]) and in four service settings (substance use disorder treatment, child welfare, education, and specialty mental health). Associations between the sustainment measure and sustainment leadership, sustainment climate, and attitudes towards the adoption and use of each of the EBPs were assessed to confirm construct validity. RESULTS: Three items for the Provider REport of Sustainment Scale (PRESS) were assessed for measuring the core component of sustainment: continued use of the EBP. Internal consistency reliability was high. The scale indicated fit to the Rasch measurement model with no response dependency, ordered thresholds, no differential item functioning, and supported unidimensionality. Additionally, construct validity evidence was provided based on the correlations with related variables. CONCLUSION: The PRESS measure is a brief, three-item measure of sustainment that is both pragmatic and useable across different EBPs, provider types, and settings. The PRESS captures frontline staffs' report of their clinic, team, or agency's continued use of an EBP. Future testing of the PRESS for concurrent and predictive validity is recommended.

COVID-related work changes, burnout, and turnover intentions in mental health providers: A moderated mediation analysis.

Objective: The novel coronavirus disease (COVID-19) has drastically impacted the provision of mental health services. Changes required of providers were substantial and could lead to increased burnout and, subsequently, increased turnover intentions. This study examined burnout experienced by mental health services providers in the context of COVID-19 and through the lens of the job demands-resources (JD-R) model. We examined the effects of work changes on burnout and subsequent turnover intentions, and how job and personal resources may have buffered the extent to which work changes due to COVID-19 impacted burnout. Methods: Service providers (n = 93) from six community mental health centers (CMHCs) in one Midwestern state in the United States completed surveys as part of service contracts to implement evidence-based practices. Path analysis tested the unconditional indirect relations between work changes and turnover intentions through burnout. Moderated mediation determined whether the indirect effect of work changes on turnover intentions via burnout varied in strength by job and personal resources. Results: Work changes had a significant indirect effect on turnover intentions through burnout ( ß ^ = .140, 95% CI = .072, .217). This indirect effect varied as a function of two job resources, organizational trust and perceived organizational support. Conclusions and Implications for Practice: Burnout was relatively low only when work changes were low and job resources levels high. When work changes were high, burnout was similarly high across levels of job resources. To minimize burnout, organizations should limit task, setting, and team-related work changes to the extent possible. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

An observational study of the impact of COVID-19 and the rapid implementation of telehealth on community mental health center providers.

BACKGROUND: The COVID-19 pandemic has remarkably altered community mental health service delivery through the rapid implementation of telehealth. This study reports provider perspectives on the impact that COVID-19 and the transition to telehealth had on their work and their ability to deliver evidence-based practices (EBPs). METHODS: Providers (n = 93) completed online surveys with quantitative measures and open-ended items exploring their reactions to COVID-19 and to the transition to providing services via telehealth. RESULTS: Perceptions of personal risk and rumination around COVID-19 were low, while telehealth was viewed positively by providers. Three major themes emerged regarding the major impacts of COVID-19 on work: (1) the altered nature of interactions between patient/client and provider due to telehealth implementation, (2) changes in provider expectations regarding productivity, and (3) challenges maintaining work-life balance. In regard to the major impacts of COVID-19 on EBP delivery, three themes emerged: (1) increased difficulty delivering certain therapies via telehealth, (2) potential limitations to session confidentiality, and (3) challenge of engaging children in telehealth. CONCLUSIONS: In the context of the COVID-19 pandemic, community mental health providers continued to engage with clients and deliver EBPs while navigating a number of changes related to the rapid transition to and implementation of telehealth. This study highlights the need for further work on what supports providers need to effectively engage with clients and deliver EBPs via telehealth, and has implications for how telehealth is sustained or de-implemented post-COVID-19.

Racial and Ethnic Differences in Depressive Profiles of Child Welfare-Involved Families Receiving Home Visitation Services.

OBJECTIVE: Profiles of depressive symptoms were identified among Hispanic, Black, and White parents involved in the child welfare service system, including changes in symptoms over time. METHODS: Participants (N=2,109) were parents receiving SafeCare, a home visitation intervention provided in a large, diverse child welfare system. Depressive symptoms were assessed with the Centers for Epidemiological Studies Depression Scale at baseline and at approximately every sixth home visit, up to a total of four times. Univariate tests examined the relationship between baseline symptoms, race-ethnicity, and service participation. Latent class growth analyses identified trajectories of depressive symptoms during participation in child welfare services. RESULTS: Participation in services was affected by depressive symptoms. Forty percent of parents did not remain long enough in the program to complete a second CES-D assessment, and those who reported more symptoms at baseline were significantly less likely to do so. Among parents who engaged in services, distinct profiles of depressive symptoms emerged that differed by race-ethnicity. For non-Hispanic Black parents, no changes in depressive symptoms over time were noted, regardless of level of severity at baseline. Parents with the highest levels of symptoms did not improve over time. CONCLUSIONS: Despite receipt of supportive and recovery-oriented services specifically focused on empowering child welfare-involved parents, many experienced elevated depressive symptoms. Integration of child welfare and community mental health systems may improve both service engagement and mental health among child welfare-involved families.

Advancing the pragmatic measurement of sustainment: a narrative review of measures.

BACKGROUND: Sustainment, an outcome indicating an intervention continues to be implemented over time, has been comparatively less studied than other phases of the implementation process. This may be because of methodological difficulties, funding cycles, and minimal attention to theories and measurement of sustainment. This review synthesizes the literature on sustainment measures, evaluates the qualities of each measure, and highlights the strengths and gaps in existing sustainment measures. Results of the review will inform recommendations for the development of a pragmatic, valid, and reliable measure of sustainment. METHODS: A narrative review of published sustainment outcome and sustainability measures (i.e., factors that influence sustainment) was conducted, including appraising measures in the Society of Implementation Research Collaboration (SIRC) instrument review project (IRP) and the Dissemination and Implementation Grid-Enabled Measures database initiative (GEM-D&I). The narrative review used a snowballing strategy by searching the reference sections of literature reviews and definitions of sustainability and sustainment. Measures used frequently and judged to be comprehensive and/or validated by a team of implementation scientists were extracted for analysis. RESULTS: Eleven measures were evaluated. Three of the included measures were found in the SIRC-IRP, three in the GEM-D&I database, (one measure was in both databases) and six were identified in our additional searches. Thirteen constructs relating to sustainment were coded from selected measures. Measures covered a range of determinants for sustainment (i.e., construct of sustainability) as well as constructs of sustainment as an outcome. Strengths of the measures included, development by expert panels knowledgeable about particular interventions, fields or contexts, and utility in specific scenarios. A number of limitations were found in the measures analyzed including inadequate assessment of psychometric characteristics, being overly intervention or context specific, being lengthy and/or complex, and focusing on outer context factors. CONCLUSION: There is a lack of pragmatic and psychometrically sound measures of sustainment that can be completed by implementation stakeholders within inner context settings (e.g., frontline providers, supervisors).

An Observational Study of the Impact of COVID-19 and the Transition to Telehealth on Community Mental Health Center Providers.

BACKGROUND: The COVID-19 pandemic has swiftly and remarkably altered community mental health service delivery and evidence-based practice (EBP) implementation. This study reports provider perspectives on the impact that COVID-19 had on their work and EBP implementation. METHODS: Providers (n = 93) completed online surveys with quantitative measures and open-ended items targeting their responses and/or reactions to COVID-19, and to the transition to providing services via telehealth. RESULTS: Perceptions of personal risk and rumination around COVID-19 were low, while telehealth was viewed positively by providers. Three major themes emerged regarding the major impacts of COVID-19 on work: 1) the altered nature of interactions between patient/client and provider, 2) changes in provider expectations regarding productivity, and 3) challenges maintaining work-life balance. In regard to the major impacts of COVID-19 on EBP implementation, three themes emerged: 1) increased difficulty delivering certain therapies via telehealth, 2) potential limitations to session confidentiality, and 3) challenge of engaging children in telehealth. CONCLUSIONS: In the context of the COVID-19 pandemic, community mental health providers continued to engage with clients and implement EBPs while navigating a number of changes related to the transition to telehealth. This study highlights the need for further work on what supports providers need to effectively engage with clients and deliver EBPs via telehealth and has implications for how telehealth is sustained or de-implemented in response to COVID-19.

Enhancing primary care services for diverse sexual and gender minority populations: a developmental study protocol.

INTRODUCTION: Compared with heterosexual, cisgender populations, sexual and gender minority (SGM) people are more likely to suffer from serious health conditions and insufficient access to health services. Primary care is at the frontlines of healthcare delivery; yet, few clinics have resources or mechanisms in place to meet SGM patient needs. This developmental study protocol focuses on reducing health disparities among SGM patients by identifying, adapting and developing SGM practice guidelines/recommendations and implementation strategies for primary care clinics in urban and rural New Mexico. Using input from patients, healthcare advocates and providers, and researchers, the study will pilot a practice parameter and implementation toolkit to promote SGM-specific cultural competence at multiple service delivery levels. METHODS AND ANALYSIS: We will recruit providers/staff from four Federally Qualified Health Centers (FQHCs) serving ethnically and geographically diverse communities. Incorporating the Implementation of Change Model and an intersectionality perspective, data collection includes a systematic review of SGM-specific practice guidelines/recommendations, focus groups and semistructured interviews, quantitative surveys and the Nominal Group Technique (NGT) with providers/staff. We will categorise guidelines/recommendations identified through the review by shared elements, use iterative processes of open and focused coding to analyse qualitative data from focus groups, interviews and the NGT, and apply descriptive statistics to assess survey data. Findings will provide the foundation for the toolkit. Focus groups with SGM patients will yield supplemental information for toolkit refinement. To investigate changes in primary care contexts following the toolkit's pilot, we will undertake systematic walkthroughs and document review at the FQHCs, analysing these data qualitatively to examine SGM inclusiveness. The structured data-informed Plan-Do-Study-Act method will enable further revision of the toolkit. Finally, focus groups, interviews and quantitative surveys with providers/staff will highlight changes made in the FQHCs to address SGM patient needs, barriers to sustainment of changes, satisfaction, acceptability, usability and feasibility of the toolkit. ETHICS AND DISSEMINATION: The study has been reviewed and approved by the Pacific Institute for Research and Evaluation Institutional Review Board. Informed consent will be obtained from all participants before their involvement in research activities begins. Study results will be actively disseminated through peer-reviewed journals, conference presentations, social media and the internet, and community/stakeholder engagement activities.

Comparative case studies in integrated care implementation from across the globe: a quest for action.

BACKGROUND: Integrated care is the coordination of general and behavioral health and is a highly promising and practical approach to improving healthcare delivery and patient outcomes. While there is growing interest and investment in integrated care implementation internationally, there are no formal guidelines for integrated care implementation applicable to diverse healthcare systems. Furthermore, there is a complex interplay of factors at multiple levels of influence that are necessary for successful implementation of integrated care in health systems. METHODS: Guided by the Exploration, Preparation, Implementation, Sustainment (EPIS) framework (Aarons et al., 2011), a multiple case study design was used to address two research objectives: 1) To highlight current integrated care implementation efforts through seven international case studies that target a range of healthcare systems, patient populations and implementation strategies and outcomes, and 2) To synthesize the shared and unique challenges and successes across studies using the EPIS framework. RESULTS: The seven reported case studies represent integrated care implementation efforts from five countries and continents (United States, United Kingdom, Vietnam, Israel, and Nigeria), target a range of clinical populations and care settings, and span all phases of the EPIS framework. Qualitative synthesis of these case studies illuminated common outer context, inner context, bridging and innovation factors that were key drivers of implementation. CONCLUSIONS: We propose an agenda that outlines priority goals and related strategies to advance integrated care implementation research. These goals relate to: 1) the role of funding at multiple levels of implementation, 2) meaningful collaboration with stakeholders across phases of implementation and 3) clear communication to stakeholders about integrated care implementation. TRIAL REGISTRATION: Not applicable.

Validation of the Implementation Climate Scale (ICS) in substance use disorder treatment organizations.

BACKGROUND: One critical factor in the implementation of evidence-based practice (EBP) in substance use disorder treatment organizations is an inner organizational context that clearly supports implementation efforts. The Implementation Climate Scale (ICS) has been developed to allow researchers and organizations to assess climate for EBP implementation in health and allied health service organizations. The ICS consists of 18 items and measures six dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection for EBP, and selection for openness. The ICS was initially developed in a mental health context; thus, the goal of this study was to provide initial validation of the ICS in substance use disorder (SUD) treatment settings. METHODS: Confirmatory factor analysis (CFA) was used to assess the psychometric functioning of the ICS using survey data from 326 providers in 65 teams in SUD treatment programs. Cronbach's alpha was examined to assess internal consistency of the ICS, and individual and team level construct-based validity was examined by comparing its correlations with service climate, molar climate, and organizational change. RESULTS: We found evidence for the reliability, factor structure, and validity of the ICS in SUD services. The psychometric functioning of the ICS in SUD treatment settings was comparable to that found in mental health contexts. CONCLUSIONS: The ICS is a brief and pragmatic tool for researchers to better understand a critical antecedent for implementation effectiveness in SUD treatment and for organizational leaders in SUD treatment organizations to evaluate the extent to which providers perceive that their organization supports EBP implementation.

Beyond checkboxes: A qualitative assessment of physicians' experiences providing care in a patient-centred medical home.

RATIONALE, AIMS, AND OBJECTIVES: The patient-centred medical home (PCMH) is an innovative approach to health care reform. Despite a well-established process for recognizing PCMH practices, fidelity to, and/or adaptation of, the PCMH model can limit health care and population health improvements. This study explored the connection between fidelity/adaptation to the PCMH model with implementation successes and challenges through the experiences of family and internal medicine PCMH physicians. METHODS: Interviews were conducted at two academic PCMH clinics with faculty and resident physicians. Data were transcribed and coded on the basis of an a priori code list. Together, the authors reviewed text and furthered the analysis process to reach final interpretation of the data. RESULTS: Ten faculty and nine resident physicians from the Family Care Centre (FCC; n = 11) and the Internal Medicine Clinic (IMC; n = 8) were interviewed. Both FCC and IMC physicians spoke positively about their clinic's adherence to the PCMH model of enhanced access to care, coordinated/integrated care, and improvements in quality and safety through data collection and documentation. However, physicians highlighted inadequate staffing and clinic hours. FCC physicians also discussed the challenge of providing high-quality care amidst differences in coverage between payers. CONCLUSION: There remains significant variability in PCMH characteristics across the United States and Canada. This qualitative analysis uncovered factors contributing to fidelity/adaptation to the PCMH model in two academic PCMH clinics. For the PCMH to achieve the Triple Aim promise of improved patient health and experience at a reduced cost, policy must support fidelity to core elements of the PCMH.